Scripts to find badly scaling phases - #349
Merged
Merged
Conversation
scaling-extract.py reads one run of a branch from the result database (the latest of every Scalable* library by default), downloads and caches every model's .err and .sim log, and stores in a local sqlite3 file what the database does not have: omc's execStat line of every pass of the translation with its time, allocations and system size, the runtime's LOG_STATS timers and counters, the backend's model statistics and the exit status of a killed omc. A stale omdb tunnel makes it fail with a message rather than hang. scaling-analyze.py splits each family into series that vary one name parameter, takes the backend's equation count as the size (the grids of ScalableTestGrids have N^2 equations per N), fits log-log exponents over the largest sizes and reports every phase, pass and timer that grows faster than size^1.5 and costs more than a second, the models that did not simulate, and a summary per pass. Assisted-by: Claude Fable 5.1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
scaling-extract.py reads one run of a branch from the result database (the latest of every Scalable* library by default), downloads and caches every model's .err and .sim log, and stores in a local sqlite3 file what the database does not have: omc's execStat line of every pass of the translation with its time, allocations and system size, the runtime's LOG_STATS timers and counters, the backend's model statistics and the exit status of a killed omc. A stale omdb tunnel makes it fail with a message rather than hang.
scaling-analyze.py splits each family into series that vary one name parameter, takes the backend's equation count as the size (the grids of ScalableTestGrids have N^2 equations per N), fits log-log exponents over the largest sizes and reports every phase, pass and timer that grows faster than size^1.5 and costs more than a second, the models that did not simulate, and a summary per pass.
Assisted-by: Claude Fable 5.1